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METHODS FOR CONSTRUCTING MULTIMEDIA DATABASE AND 
PROVID ING MULTIMEDIA-SEARCH SERVICE AND APPARATUS 
" THEREFOR 



5 

Technical Field 

The present invention relates to a method and a system for 
constructing a multimedia database for indexing multimedia data, and a 
method and a system for providing a search service using the multimedia 
10 database. 

Background Art 

In accordance with the increase in the storage capacity of 
computers, the wide use of the Internet, and the spread of high-speed 

is communication networks, great attention has been paid to multimedia 
content including a moving picture and voice data having a large file size. 

Accordingly, many search service providers, who used to be 
capable of providing only text as search results/have started to provide 
search services for multimedia data to consumers. 

20 According to a conventional multimedia content search service, 

each multimedia file is stored in a multimedia database, and the 
multimedia database is generally constructed by using an 
entity-relationship model for developing a relationship-oriented database 
or an object-oriented analysis and a design model for developing an 

25 object-oriented database. Accordingly, when a user types in a keyword 
for multimedia data that he or she wants to search for, a multimedia file 
corresponding to the typed-in keyword is drawn from the multimedia 
database and presented to the user. 

However, if a plurality of multimedia files are obtained as search 

30 results, the user has no option but to execute all the multimedia flies in 

1 
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order to find, among the multimedia files, a multimedia data file that the 
user has been looking for. Therefore, the conventional multimedia 
content search service costs the user a considerable amount of time and 
great effort. 

5 

Disclosure of the Invention 

Accordingly, the present Invention provides a method of 
constructing a multimedia database. 

The present invention also provides a system for constructing a 
10 multimedia database. 

The present invention also provides a method of providing a 
multimedia data search service. 

The present invention also provides a system for providing a 
multimedia data search service. 
15 The present invention also provides a method of purchasing 

multimedia content 

The present invention also provides a system for purchasing 
multimedia content. 

According to an aspect of the present invention, there is provided 
20 a method of constructing a multimedia database method. The method 
Includes (a) receiving a start point and an end point of each first 
semantic unit of multimedia data, which is a smallest unit for searching 
for multimedia data, (b) receiving a keyword for each first semantic unit, 
(c) receiving a start point and an end point of each second semantic unit 
25 of the multimedia data including at least one first semantic unit, and (d) 
storing a keyword together with location information of its corresponding 
first semantic unit and second semantic unit. 

According to another aspect of the present invention, there is 
provided a system for constructing a multimedia database. The system 
30 Includes a multimedia database which stores multimedia data, a keyword 
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database which stores keywords necessary for searching for the 
multimedia data, location information of each first semantic unit of the 
multimedia data, which Is a smallest unit for searching for multimedia 
data, and location information of each second semantic unit of the 

5 multimedia data, which includes at least one first semantic unit, an input 
unit which receives the location information of each first semantic unit, 
including a start point and an end point, the location information of each 
second semantic unit, including a start point and an end point, and the 
keywords, and a control unit which receives the location information of 

10 each first semantic unit, the location information of each second 
semantic unit, and the keywords from the input unit and stores the 
keywords in the keyword database together with their corresponding first 
and second semantic units' location Information. 

According to another aspect of the present invention, there is 

15 provided a method of constructing a multimedia database. The method 
includes (a) setting a length of each first semantic unit of multimedia data, 
which is a smallest unit for searching for multimedia data according to a 
user's input, (b) extracting a keyword from each first semantic unit using 
a predetermined method, (c) setting a length of each second semantic 

20 unit of the multimedia data including at least one first semantic unit 
according to the user's input, and (d) storing the extracted keyword with 
its corresponding first semantic unit and second semantic unit. 

According to another aspect of the present invention, there is 
provided a system for constructing a multimedia database. The system 

25 Includes a multimedia database which stores multimedia data, a keyword 
database which stores keywords necessary for searching for the 
multimedia data, location information of each first semantic unit of the 
multimedia data, which is a smallest unit for searching for multimedia 
data, and location Information of each second semantic unit of the 

30 multimedia data, which includes at least one first semantic unit, a 
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keyword extraction unit which extracts keywords from the multimedia 
data using a predetermined method, and a control unit which divides the 
multimedia data into first semantic units and second semantic units and 
. stores keywords in the keyword database together with their 

5 corresponding first and second semantic units' location information. 

According to another aspect of the present Invention, there is 
provided a method of providing a multimedia data search service using a 
system for providing a multimedia data search service, including a 
multimedia database which stores multimedia data, and a keyword 

10 database which stores keywords necessary for searching for the 
multimedia data, location information of each first semantic unit of the 
multimedia data, which is a smallest unit for searching for multimedia 
data, and location information of each second semantic unit of the 
multimedia data, Including at least one first semantic unit The method 

is includes (a) receiving keywords necessary to search for multimedia data, 
(b) allowing a user to select a search unit level from between a first 
semantic unit and a second semantic unit, (c) searching for multimedia 
data of the received search unit level whose keywords match the 
received keyword, and (d) outputting Information of a searched semantic 

20 unit of the received search unit level, linking with the search semantic 
unit in the multimedia database. 

According to another aspect of the present invention, there is 
provided a system for providing a multimedia data search service. The 
system includes a multimedia database which stores multimedia 

25 database, a keyword database which stores keywords necessary for 
searching for the multimedia data, location Information of each first 
semantic unit of the multimedia data, which is a smallest unit for 
searching for multimedia data, and location Information of each second 
semantic unit of the multimedia data, which Includes at least one first 

30 semantic unit, an Input unit which receives a keyword and a search unit 
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level a user, a control unit which searches the keyword database for a 
keyword that matches the received keyword, provides links between 
resulting search results and places in the multimedia database where the 
search results are stored, and outputs some of the search results 

5 " selected by the user and a display unit which displays the searched 
results obtained by the control unit. 

According to another aspect of the present invention, there is 
provided a method of constructing a multimedia database. The method 
includes (a) receiving location information of each semantic unit of 

10 multimedia data, which is a smallest unit for searching for multimedia 
data, (b) receiving a keyword for each semantic unit, and (c) storing 
keywords together with their corresponding semantic unit" location 
Information. 

According to another aspect of the present invention, there is 

15 provided a system for constructing a multimedia database. The system 
includes a multimedia database which stores multimedia data, a keyword 
database which stores keywords necessary for searching for the 
multimedia data and location information of each semantic unit, which is 
a smallest unit for searching for multimedia data, an input unit which 

20 receives the location information of each semantic unit, including a start 
point and an end point, and the keywords, and a control unit which 
receives the location information of each semantic unit from the input 
unit and the keywords and stores the keywords in the keyword database 
together with their corresponding semantic unifs location information. 

25 According to another aspect of the present invention, there is 

provided a method of constructing a multimedia database. The method 
includes (a) receiving a length of each semantic unit of multimedia data, 
which is a smallest unit for searching for multimedia data, (b) extracting a 
keyword from each semantic unit of the multimedia data, and (c) storing 

30 keywords together with their corresponding semantic unifs location 
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information. 

According to another aspect of the present invention, there Is 
provided a system for constructing a multimedia database. The system 
includes a multimedia database which stores multimedia data, a keyword 
5 database which stores keywords necessary for searching for the 
multimedia data and location information of each semantic unit, which is 
a smallest unit for searching for multimedia data, a keyword extraction 
unit which extracts keywords from the multimedia data using a 
predetermined method, and a control unit which divides the multimedia 
10 data Into semantic units having a predetermined length and stores the 
extracted keywords together with their corresponding semantic unifs 
location information. 

According to another aspect of the present invention, there is 
provided a method for providing a multimedia data search service using 
15 a search system including a multimedia database which stores 
multimedia data and a keyword database which stores keywords 
necessary for searching for the multimedia data and location information 
of each semantic unit, which is a smallest unit for searching for 
multimedia data. The method includes inputting a keyword for 
20 searching for multimedia data, (b) searching for a semantic unit of a 
selected search unit level having the same keyword as the Input keyword, 
and (c) linking resulting search results to their locations in the multimedia 
database and presenting the search results to a user. 

According to another aspect of the present invention, there is 
25 provided a system for providing a multimedia data search service. The 
system includes a multimedia database which stores multimedia data, a 
keyword database which stores keywords necessary for searching for 
the multimedia data and location information of each semantic unit, 
which Is a smallest unit for searching for multimedia data, an input unit 
30 which receives a keyword from a user, a control unit which searches the 
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keyword database for a keyword that matches the received keyword and 
outputs resulting search results with links to their locations in the 
multimedia database, and a display unit which displays the searched 
results obtained by the control unit. 
5 According to another aspect of the present invention, there Is 

provided a method of constructing a multimedia database. The method 
includes (a) a user accessing a system, (b) allowing the user to 
designate address information of a multimedia data file desired to be 
executed by the user, (c) executing the multimedia data file by accessing 
10 a server where the multimedia data file is stored according to the 
designated address Information, (d) receiving and setting a start time and 
an end time of each first semantic unit of the multimedia data file, which 
is a smallest unit for searching for multimedia data while executing the 
multimedia data file, and receiving representative Information of each 
15 firet semantic unit, and (e) storing the representative information of each 
first semantic unit together with the start time and end time of each first 
semantic unit and the address information of the multimedia data file. 

According to another aspect of the present invention, there is 
provided a system for constructing a multimedia database. The system 
20 includes an input and output unit which allows a user to access a system, 
receives address information of a multimedia data file to be executed by 
the user, a start time and an end time of each first semantic unit of the 
multimedia data file, and representative information of each first 
semantic unit and allows the user to transmit data to or receive data from 
25 the server where the multimedia data file is stored, a keyword database 
which stores the representative information of each first semantic unit 
with the start time and end time of each first semantic unit and the 
address information of the multimedia data file, and a control unit which 
executes the multimedia data file by accessing the server where the 
30 multimedia data file Is stored in response to an input from the user using 
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the input and output unit, receives the start time and end time and the 
representative Information of each first semantic unit in response to the 
input from the user and stores the received information in the keyword 
database together with the address information of the multimedia data 
5 file, and executes a predetermined first semantic unit of the multimedia 
data file using the address information of the multimedia data file and the 
start time and end time of the predetermined first semantic unit when a 
request for searching for and reproducing the predetermined first 
semantic unit is issued by the user. 
10 According to another aspect of the present Invention, there is 

provided a method of purchasing multimedia content from a multimedia 
content owner using a predetermined purchasing system. The method 
includes (a) informing a user of purchasable multimedia contents and 
allowing the user to select multimedia content to be purchased, (b) 
15 executing the selected multimedia content using address information of 
the multimedia content stored in the purchasing system, (c) allowing the 
user to set a start time and an end time of each first semantic unit of the 
multimedia content, which Is a smallest unit for purchasing the 
multimedia, content, while executing the multimedia content, (d) storing 
20 the start time and end time of each first semantic unit of the multimedia 
content with the address information of the multimedia content, (e) 
calculating a rate for a first semantic unit according to predetermined 
standards, and (f) generating an execution file capable of executing a 
first semantic unit of the multimedia content purchased by the user using 
25 the start time and end time of the first semantic unit and the address 
information of the multimedia content stored in the purchasing system 
and providing information to which the execution file is linked. 

According to another aspect of the present Invention, there Is 
provided a system for purchasing multimedia content from a multimedia 
30 content owner using a predetermined purchasing system. The system 
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includes an input and output unit which allows a user to select 
multimedia content including a first semantic unit to be purchased and to 
set a start time and an end time of the first semantic unit, a keyword 
database which stores the start time and end time of the first semantic 

5 unit together with address information of multimedia contents that can be 
purchased using the purchasing system, a controller which executes the 
selected multimedia content using the address information stored In the 
keyword database, stores the start time and end time of the first 
semantic unit In the keyword database in response to an input from the 

10 user using the input and output unit, generates an execution file for 
executing the first semantic unit using the address information of the 
selected multimedia content and the start time and end time of the first 
semantic unit, and provides link information to which the execution file is 
linked, and a rate calculation unit which calculates a rate for the first 

15 semantic unit according to predetermined standards. 

According to another aspect of the present invention, there is 
provided a computer-readable recording medium on which a program 
enabling any of the above-described methods Is recorded. 

20 Brief Description of the Drawings 

FIG. 1 is a schematic diagram Illustrating the basic concepts of 
the present invention; 

FIG. 2 is a flowchart of a method of constructing a multimedia 
database according to a preferred embodiment of the present invention; 
25 FIG. 3 is a block diagram of a system for constructing a 

multimedia database according to a preferred embodiment of the present 
invention; 

FIG. 4 is a flowchart of a method of constructing a multimedia 
database according to another preferred embodiment of the present 
30 Invention; 

9 
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FIG. 5 is a block diagram of a system for constructing a 
multimedia database according to another preferred embodiment of the 
present invention; 

FIG. 6 is a flowchart of a method of providing a multimedia data 
5 search service according to a preferred embodiment of the present 
invention; 

FIG. 7 is a block diagram of a system for providing a multimedia 
data search service for providing a multimedia data search according to 
a preferred embodiment of the present Invention; 
10 FIG. 8 is a diagram Illustrating a search window according to a 

preferred embodiment of the present Invention; 

FIGS. 9A through 9C are diagrams illustrating examples of 
multimedia data divided into semantic units according to a preferred 
embodiment of the present Invention; 
15 FIG. 10 Is a block diagram of a system for constructing a 

multimedia database through the Internet according to a preferred 
embodiment of the present invention; 

FIG. 1 1 is a flowchart of a method of constructing a multimedia 
database through the Internet according to a preferred embodiment of 
20 the present invention; 

FIG. 12 is a block diagram of a system for purchasing multimedia 
content through the Internet according to a preferred embodiment of the 
present invention; and 

FIG. 13 is a flowchart of a method of purchasing multimedia 
25 content through the Internet according to a preferred embodiment of the 
present invention. 

Best mode for carrying out the Invention 

Hereinafter, the present invention will be described in greater 
30 detail with reference to the accompanying drawings in which preferred 
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embodiments of the present invention are shown. 

RG. 1 Is a schematic diagram Illustrating the basic concepts of 
the present invention. The present invention Is characterized by 
dividing multimedia data Into predetermined semantic units and allotting 
5 an appropriate keyword to each semantic unit In addition, the present 
invention is capable of presenting exactly what a user wants to search for 
as a search result. 

In FIG. 1, an action (111, 112, or 113), a module (121 or 122), 
and a situation (131) represent a smallest unit, a semantic unit including 
10 at least one action, and a semantic unit Including at least one module, 
respectively. 

In particular, FIG. 1 illustrates a bitstream of a moving Image. 
The bitstream is temporally divided into (marked off at predetermined 
intervals into) an action 1 (1 1 1 ), an action 2 (1 12), and an action 3 (113). 

15 The actions 1 through 3 (111 through 113) are associated with their 
respective keywords. 

Modules 121 and 122 are semantic units which are at least as 
long as one action. The modules 121 and 122 directly Inherit keywords 
from their corresponding actions. For example, as shown In FIG. 1 , the 

20 action 1 (111) has keywords a, b, and c, and the action 2 (112) has 
keywords c, d, and f. Accordingly, the module 1(121), including the 
action 1 (111) and the action 2 (112), directly inherits the keywords a, b, 
c, d, and f. 

A situation 131 is a semantic unit, which is at least as long as one 
25 module. The situation 131 inherits keywords from its corresponding 
modules, i.e., the modules 121 and 122. 

A start point and an end point of each action, each module, and 
each situation may be arbitrarily set by a database constructor who 
inputs keywords or may be automatically set and renewed at intervals of 
30 a predetermined amount of time. 
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A method of constructing the multimedia database shown in FIG. 
1 will be described In greater detail in the following paragraphs. FIG. 2 
is a flowchart of a method of constructing a multimedia database 
according to a preferred embodiment of the present invention, and FIG. 

5 3 is a block diagram of a system for constructing a multimedia database 
according to a preferred embodiment of the present invention. Steps of 
the method for constructing a multimedia database shown in FIG. 2 may 
be arranged in a different order other than the one shown in FIG. 2. 

Referring to FIG. 3, the system for constructing a multimedia 

10 database includes a multimedia database 304 which stores multimedia 
data, a keyword database 303 which stores keywords necessary for 
searching for the multimedia data, an input unit 301 which receives 
keywords, and a control unit 302 which controls an entire process of 
constructing a multimedia database. 

15 By using the input unit 301, a multimedia database constructor 

(hereinafter, referred to as the constructor) inputs location information of 
each first semantic unit (for example, each action), which is a smallest 
unit for searching for multimedia data, in step 201. Here, the location 
information of each first semantic unit indicates information on where a 

20 start point and an end point of each first semantic unit of multimedia data 
are located in the multimedia database 304. 

Here, first semantic units, I.e., actions, may be continuously or 
discontinuous^ arranged. In other words, an action does not need to 
begin where its previous one ends. Therefore, the constructor may 

25 extract only desired portions from the multimedia data and then set the 
extracted portions as first semantic units, i.e., actions. 

When the start point and end point of each first semantic unit of 
the multimedia data are input, the control unit 302 extracts location 
information of each first semantic unit from the muHimedia database 304. 

30 ASF files (*.asf) are generally used because they can provide a function 
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of extracting location information in a multimedia file from a marked point. 
However, other files can serve the same function as the ASF files with 
the help of predetermined programs, and thus there is no restriction 
imposed on the types of files that can be used in the present invention. 

5 The constructor inputs one keyword or more for each of the first 

semantic units in step 202. Here, one or more keywords may be 
allotted to each of the first semantic units. The constructor may 
manually input the keyword(s) by directly typing in the keyword(s) or 
using a speech recognition technique where speech can be recognized 

10 as text 

Alternatively, the constructor may set In advance a variety of 
pieces of information, such as images, voice, and text, as keywords. 
Then, the ones that match the preset information in the multimedia data 
are automatically extracted as keywords. 

is In the case of moving picture data or voice, speech can be 

extracted and transformed into text data using a speech recognition 
technique, and then only predetermined parts of the extracted speech, 
such as nouns or pronouns, can be extracted as keywords. A method 
of extracting predetermined functional parts of speech using a speech 

20 recognition technique and a database including a dictionary is well 
known to the one skilled in the art, and thus its description will be 
omitted. 

The controller 302 stores the Input keyword(s) in the keyword 
database 303 along with its (their) corresponding first semantic unifs 
25 location information in the multimedia database 304. 

Before storing keywords In the keyword database 303, the 
keywords may be classified into several categories depending on what 
they concern, such as people, objects, places, or moments of time. 
Then, keywords belonging to one category are stored in the keyword 
30 database 303 separately from the ones belonging to another category. 
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Thus, The keywords stored In the keyword database 303 can be 
presented to a user on a category-by-category basis. For example, a 
series of keywords only concerning places can be presented to the user 
at the same time, and thus the user can easily and conveniently figure 
5 out places where a predetermined moving picture corresponding to the 
presented keywords has been taken based upon the presented 
keywords. 

Preferably, the constructor may give a title to each of the first 
semantic units of the multimedia data and inputs the title of each of the 

10 first semantic units into the keyword database 303 along with its 
corresponding keyword(s), a process which is not shown in FIG. 2. By 
doing so, it is possible to exactly and conveniently figure out what each 
of the first semantic units is about based upon its corresponding title and 
keyword(s). For example, supposing that a user types in words "Jesus' 

15 and 'cross' as keywords for the purpose of searching a movie title 
concerning a biblical story for a predetermined scene showing Jesus 
nailed upon the Cross, a plurality of semantic units showing a variety of 
scenes, such as a scene showing Jesus climbing up a hill at Golgotha 
with the Cross on the back, are simultaneously popped out as search 

20 results. At this time, if the plurality of semantic units are presented to a 
user with their corresponding title, the user can easily Identify the one 
that he or she desires to search for from among the plurality of semantic 
units. 

Keywords are preferably stored in the keyword database 303 so 
25 that keywords having similar meanings or indicating the same thing can 
be associated with one another. For example, let us assume that 
•Jesus', the son of the God', and the Messiah' are designated as 
keywords of semantic units 1 through 3, respectively. Even though the 
semantic units 1 through 3 seemingly have different keywords, their 
30 keywords indicate the same thing, and thus when a user types in 'Jesus' 
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rather than 'the son of the God' or the Messiah', the semantic units 1 
through 3 corresponding to all the three keywords can be searched for 
as search results. 

Here, in order to store keywords with their associated keywords in 
5 the keyword database 303, the constructor may set keywords that need 
to be associated before or after inputting a variety of keywords for every 
semantic unit of predetermined multimedia data. Thus, keywords 
having similar meanings or Indicating the same thing can be stored 
together In the keyword database 303 by the control unit 302. 
10 Alternatively, a similar keyword database where keywords having similar 
meanings or indicating the same thing are stored together may be 
additionally provided, and then when a keyword is input and the one that 
exactly matches the input keyword exists among the keywords stored in 
the similar keyword database, the input keyword and its similar keywords 
15 are stored together in the keyword database 303. 

A multimedia database can be completed by only performing 
steps 201 and 202 that have been described so far. However, In the 
present invention, it Is preferable that the constructor inputs information 
on a start point and an end point of each second semantic unit including 
20 at least one first semantic unit in step 203. 

Let us assume that multimedia data is searched on a first 
semantic unit basis and that the keyword database 303 only stores 
information on where In the multimedia database 304 the start point of 
each first semantic unit of the multimedia data is stored. In this case, 
25 the user may not be able to figure out exactly where each first semantic 
unit obtained as a search result ends and which portion of each first 
semantic unit corresponds to the keyword that he/she has input. Thus, 
the user may have no other options but to execute the entire file 
including the searched first semantic units to find data that he/she really 
30 wants. 
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In order to prevent such inconvenience, the keyword database 

303 stores information on where in the multimedia database 304 both 
the start point and end point of each of the first semantic units of the 
multimedia data are stored so that each search result can be executed 

5 on a first semantic unit-by-first semantic unit basis, which is preferable 
for quick search. 

In some cases, the user may not be able to figure out whether or 
not the search results are what he or she desires to search for because 
the running time of each of the search results executed on a first 
10 semantic unit-by-first semantic unit basis is too short. Therefore, the 
multimedia data is preferably marked off Into second semantic units each 
of which includes at least one first semantic unit and thus Is longer than a 
first semantic unit, and information on where in the multimedia database 

304 the start point and end point of each of the second semantic units 
15 are stored is stored in the keyword database 303. 

Alternatively, it is possible to allow the user to appreciate 
sufficiently long search results by marking off the multimedia data Into 
third semantic units, each of which includes at least one second 
semantic unit and Information on where in the multimedia database 304 

20 the start point and end point of each of the third semantic units are 
stored in the keyword database 303. 

As described above, the multimedia data may be divided into 
actions, modules, or situations, which is shown in FIGS. 9A through 9C. 
Then, information on where in the multimedia database 304 the start 

25 point and end point of the action, the module, or the situation 
corresponding to the predetermined keyword are stored is stored in the 
keyword database 303. 

Since a second semantic unit Inherits a keyword or keywords of its 
corresponding first semantic unit(s), there is no need to newly input a 

30 keyword or keywords for the second semantic unit. However, it is 
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preferable to give a title to each second semantic unit. 

A keyword and information on where in the multimedia database 
304 the start point and end point of a semantic unit corresponding to the 
keyword are stored are stored in the keyword database 303 via the 
5 controller 302. First, let us assume that the multimedia is marked off 
into first semantic units and into second semantic units. In this case, 
keywords input by the constructor are stored in the keyword database 
303, and then location information of each of the first and second 
semantic units, i.e., information on where in the multimedia database 
10 304 the start point and end point of each of the first and second 
semantic units are stored, is stored in the keyword database 303 so that 
the keywords and their corresponding first and second semantic units' 
location information can be stored together. Furthermore, a title of each 
of the first and second semantic units, a category of each of the 
is keywords, and location information of multimedia content including the 
first and second semantic units are also stored in the keyword database 
303. In a case where the multimedia content is a moving picture or 
voice, its start point and end point are stored with its corresponding 
keyword. However, in a case where the multimedia content is an image, 
20 there is no need to store the start point and end point of the image. 

The method and the system for constructing a multimedia 
database according to a preferred embodiment of the present invention 
where a constructor is required to determine the length of each semantic 
unit and input keywords for each semantic unit have been described so 
25 far with reference to FIGS. 2 and 3. However, without directly inputting 
such Information and keywords, a multimedia database can be 
established with the help of a predetermined program, which will be 
described in greater detail In the following paragraphs with reference to 
FIGS. 4 and 5. FIG. 4 is a flowchart of a method of constructing a 
30 multimedia database according to another preferred embodiment of the 
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present invention, and FIG. 5 is a block diagram of a system for 
constructing a multimedia database according to another preferred 
embodiment of the present invention. The sequence of steps of the 
method of constructing a multimedia database shown In FIG. 4 is not 
s limited to the one set forth herein. 

Referring to FIG. 5, the system for constructing a multimedia 
database includes a multimedia database 504 where multimedia data is 
stored, a keyword database 503 where keywords necessary for 
searching for the multimedia data are stored, an input unit 505 which 
10 receives the length of each semantic unit of the multimedia data and the 
keywords, a keyword extraction unit 501 which extracts the keyword from 
the multimedia data, and a control unit 502 which controls an entire 
process of constructing a multimedia database. 

A constructor sets the length of each first semantic unit and the 
15 length of each second semantic unit In steps 401 and 403. Here, a first 
semantic unit is a smallest unit for searching for multimedia data. The 
control unit 502 marks off the multimedia data at Intervals of a 
predetermined distance in consideration of the lengths set in steps 401 
and 403 and extracts information on where in the multimedia database 
20 504 marked portions of the multimedia data are stored. In a case 
where the constructor has not input any Information on the length of 
each first semantic unit and the length of each second semantic unit, 
each first semantic unit and each second semantic unit are set to have a 
predetermined length using reference length values stored in a program. 
25 In step 402, the keyword extraction unit 501 extracts a keyword 

from each first semantic unit using a predetermined method. 

Various methods can be used to extract keywords from data. As 
described above with reference to FIGS. 2 and 3, in the case of a moving 
image or voice, speech can be extracted and transformed into text data 
30 using a speech recognition technique, and then only predetermined parts 
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of the extracted speech, such as nouns or pronouns, can be extracted as 
keywords. 

Alternatively, the constructor may Input In advance keyword 
information that can be used as keywords for semantic units of 

5 predetermined multimedia data, such as Images, voices, and text. Into 
the keyword extraction unit 501 via the input unit 505 and then allots a 
predetermined keyword to semantic units of the corresponding 
multimedia data containing the same information as the keyword 
information. For example, let us assume that the constructor sets the 

10 voice and images of Jesus into the keyword extractor 501 as keyword 
information for a movie title about biblical stories so that a predetermined 
keyword "Jesus" can be given to any semantic units of the movie title 
containing the voice or images of Jesus. Then, the keyword extraction 
unit 501 extracts the keyword "Jesus" from all the semantic units of the 

is movie title where Jesus appears, and the control unit 502 stores the 
keyword "Jesus" in the keyword database 503 together with Its 
corresponding semantic unifs location information in the multimedia 
database 504, I.e., information where in the multimedia database 504 the 
corresponding semantic unit Is stored. 

20 As described above with reference to FIGS. 2 and 3, keywords 

are stored together in the keyword database 503 so that when there is a 
request for search for multimedia data corresponding to any of 
associated keywords having similar meanings, all multimedia data 
corresponding to all the associated keywords are searched for In 

25 response to the search request. 

In order to store keywords with their associated keywords in the 
keyword database 503, the constructor may set in advance 
predetermined keywords that need to be associated before or after 
inputting keywords that can represent predetermined multimedia data 

30 into the keyword database 503. Alternatively, a similar keyword 
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database where similar keywords are stored may be additionally 
provided. So, when a keyword that matches any of the keywords stored 
in the similar keyword database is input by the constructor, its similar 
keywords stored in the similar keyword database are automatically 

5 stored in the keyword database 303 together. 

Step 404 where keywords are stored in the keyword database 503 
is the same as step 204 described above with reference to FIGS. 2 and 3, 
and thus its description will be omitted, 

FIG. 6 is a flowchart of a search service providing method 

10 according to a preferred embodiment of the present invention, and FIG, 
7 is a block diagram of a search service providing system according to a 
preferred embodiment of the present invention. 

Referring to FIG. 7, the multimedia data search service providing 
system includes a multimedia database 704 where multimedia data is 

15 stored, a keyword database 703 where keywords necessary for 
searching for multimedia data are stored, an input unit 701 which allows 
a user to select semantic units or input keywords, a screen display unit 
702 which displays search results, and a control unit 705 which controls 
the entire process of providing a multimedia data search service. 

20 FIG. 8 Is a diagram illustrating a search window for inputting 

keywords necessary for searching for multimedia data. As shown in a 
semantic unit selection window 801 of FIG. 8, three different levels of 
semantic units, i.e., a situation, a module, and an action, can be selected 
so that multimedia data can be searched for on a situation~by-sltuation, 

25 module-by-module, or action-by-action basis. 

A user inputs a keyword to be searched for into a keyword Input 
window 802 in step 601. The user may directly type in the kfeywond or 
select one of keywords put on a keyword list 804 by clicking a list button 
803 next to the keyword input window 802. 

30 In step 602, the user selects the amount by which each search 
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result will be presented to the user, i.e., a semantic unit of each search 
result, among the three levels of semantic units provided In the semantic 
unit selection window 801 . 

Thereafter, a search for multimedia data corresponding to the 

5 Input keyword is carried out in step 603. The control unit 705 searches 
the keyword database 703 for the one that matches the Input keyword ■ 
and displays search results, i.e., multimedia data corresponding to the 
input keyword, and their location information in the multimedia database 
704 using the screen display unit 702. 

10 In a case where each semantic unit of multimedia data is stored in 

the multimedia data 704 with its title, the controller 705 displays the 
search results and their respective titles on a screen and creates a link 
between each of the titles and its corresponding location information in 
the multimedia database 704 so that each of the search results can be 

is reproduced from the multimedia database 704 when the user clicks Its 
title on the screen. In a case where the multimedia data is stored in the 
multimedia database 704 without titles, the control unit 705 displays all 
keywords corresponding to each of the search results so as to help the 
user figure out the content of each of the search results. 

20 The user selects one of the search results in step 604. Then, the 

controller 705 finds the location of the selected search result in the 
multimedia database 704 and reproduces the selected search result 
from the multimedia database 704 in step 605. 

The present invention can be applied to multimedia data stored in 

25 an external server through the Internet as well as multimedia database 
stored in a single independent system like the embodiments set forth 
herein. 

FIG. 10 is a block diagram of an Internet-based system for 
constructing a multimedia database according to a preferred 
30 embodiment of the present invention, and FIG. 11 is a flowchart of an 
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Internet-based method of constructing a multimedia database according 
to a preferred embodiment of the present invention. 

Referring to FIG. 10, an Internet-based system for constructing a 
multimedia database 1000 includes an input and output unit 1001. a 
5 control unit 1002. and a keyword database 1003 rather than a 
multimedia database. . 

The input and output unit 1001 receives data from and transmits 
data to a user 1030 and a server 1020 where multimedia data files are 
stored by being connected to the user 1030 and the server 1020 through 
to the Internet 1010. 

In the keyword database 1003, location information of multimedia 
files indexed by the user 1030 and other representative Information, such 
as start and end points and keywords of each semantic unit, are stored. 
The control unit 1002 accesses the server 1020 where multimedia 
15 data files are stored in response to a request input by the user 1030 
through the input and output unit 1001, and executes multimedia data 
files stored In the server 1020, receives start time information and end 
time information and representative information of each semantic unit 
corresponding to the input keywords and stores the received Information 
20 in the keyword database 1003. When the user 1030 wants a searched 
semantic unit of a multimedia data file to be reproduced, the control unit 
1002 controls a multimedia player for the multimedia data file to 
reproduce only the searched semantic unit of the multimedia data file by 
using the start time information and the end time information of the 
25 searched semantic unit and location information of the multimedia data 
file in the keyword database 1030. 

The present embodiment will be described in greater detail in the 
following paragraphs. 

The user 1030 accesses a web page provided by the 
30 Internet-based system for constructing a multimedia database 1000 In 
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step 1101. 

User identification, such as log-in, is performed, and then the user 
1030 designates a path of a multimedia data file to be indexed in step 
1102. The multimedia data file may be a file stored in a database 
5 provided In the system of the present invention, as shown in FIGS. 3, 5, 
and 7, or a file stored in the server 1020 that can be accessed through 
the Internet 1010. 

For example, in a case where multimedia data that the user 1030 
wants to index is scenes broadcasted on TV, multimedia data files can 
10 be stored in a server operating a web page of a broadcasting system. 

The user 1030 may designate such path by inputting the Internet 
address of a place where the multimedia data file is stored. 

Then, the control unit 1002 accesses a predetermined server 
according to the designated path and executes the multimedia data file 
15 stored in the server. 

While watching reproduced multimedia data, the user 1030 sets 
first semantic units, as described above with reference to FIG. 2, and 
inputs a keyword and a title for each first semantic unit in step 1 104. A 
process of setting a start point and an end point of each first semantic 
20 unit may be performed before or after a process of inputting a keyword 
for each first semantic unit. 

After the setting of the first semantic unit, the control unit 1002 
stores location information of the reproduced multimedia data and 
Information on a start point and an end point of each first semantic unit 
25 of the reproduced multimedia data in the keyword database in step 1 105. 

A process of searching for and executing a desired portion of the 
multimedia data using a multimedia database is the same as the one 
shown In FIG. 6 except for several differences therebetween, which are 
as follows. 

30 First of all, in FIG. 10, the user 1030 transmits data to and 
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receives data from external servers by accessing the Internet-based 
system for constructing a multimedia database 1000 of the present 
invention through the Internet. Second, In FIG. 10, a multimedia 
database is connected to the Internet-based system for constructing a 

5 multimedia database 1000 through the Internet. Third, in FIG. 10, an 
external server that stores a multimedia data file selected by the user 
1030 is accessed according to a path stored In the keyword 
database1003 when the user 1030 desires a search result to be 
reproduced. Fourth, only a predetermined semantic unit corresponding 

10 to a keyword selected by the user 1030 is presented to the user 1030 
using the start point and end point of the predetermined semantic unit. 

The Internet-based method of constructing a multimedia database 
of the present invention makes it possible to create a link between a 
multimedia data file containing multimedia content and a web page so 

15 that only a predetermined portion of the multimedia content can be 
executed by using the link provided on the web page. In addition, the 
Internet-based method of constructing a multimedia database of the 
present invention makes it possible to sell a specific portion of 
multimedia content contained in a multimedia data file. 

20 For example, let us assume that a newspaper company is 

attempting to provide a news report concerning a predetermined event 
together with moving pictures concerning the predetermined event 
broadcasted on TV via its web page. In the prior art, the newspaper 
company has to purchase a file containing a news program for the day of 

25 the predetermined event, decode the purchased file, extract data desired 
to be reproduced from the decoded file, encode the extracted data Into a 
file, and creating a link between the file and the web page so mat the file 
can be executed using the link. This method, however, Is very 
inconvenient and thus has not been adopted very often to execute only a 

30 portion of specific multimedia content. 
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On the other hand, the Internet-based system for constructing a 
multimedia database and method make it possible to obtain the same 
effect as the above-described conventional method with even more 
simplified processes. 

5 Hereinafter, a method and a system for creating a link between a 

web page and a predetermined portion of multimedia content contained 
in a multimedia data file using a multimedia content purchasing system 
according to a preferred embodiment of the present invention will be 
described in greater detail. A process of dividing multimedia data into 

10 first semantic units or second semantic units has already been described 
above, and thus Its description will be omitted. In the following 
description, only a case where multimedia data is divided into first 
semantic units will be focused on. 

FIG. 12 is a block diagram of a system for purchasing multimedia 

15 content through the Internet according to a preferred embodiment of the 
present invention, and FIG. 13 is a flowchart of a method of purchasing 
multimedia content through the Internet according to a preferred 
embodiment of the present invention. 

The multimedia content purchasing system according to a 

20 preferred embodiment of the present invention may be located in a 
server 1220 of a multimedia content owner, a server of a multimedia 
content purchaser 1230, or a server of another business person. If 
there are a plurality of people who are willing to buy multimedia content, 
they are provided with multimedia content via different multimedia 

25 purchasing systems. Preferably, each multimedia purchasing system 
can be used after being authorized by its owner. 

The multimedia content purchasing system according to a 
preferred embodiment of the present invention includes an input and 
output unit 1201, a control unit 1202, a keyword database 1203, and a 

30 rate calculation unit 1204. 
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In the keyword database 1203, location information of multimedia 
data files provided by a multimedia content owner is stored. Addresses 
of multimedia data files stored in the keyword database 1203 are 
preferably encrypted so that they cannot be interpreted by purchasers or 
5 other users. By doing so, multimedia content can be prevented from 
being illicitly accessed or linked. 

When a purchaser accesses the multimedia content purchasing 
system or drives the multimedia content purchasing system in his or her 
server, the control unit 1202 presents a list of multimedia contents that 
io can be purchased, among multimedia contents registered in the keyword 
database 1 203, in step 1 301 . Then, in step 1 302, the purchaser selects 
multimedia content to be purchased. 

In step 1303, the control unit 1202 accesses a server where the 
selected multimedia content is stored according to path Information of 
is the selected multimedia content, draws the selected multimedia content 
from the corresponding server, and executes the drawn multimedia data. 
For executing the multimedia data, a multimedia player may be further 
included in the multimedia content purchasing system, or a multimedia 
player possessed by the purchaser may be used. 
20 In step 1304, the control unit 1202 receives representative 

information of a semantic unit desired by the purchaser, such as 
keywords, and a title of the semantic unit, a start time and an end time 
and stores the representative information in the keyword database 1203. 
If the purchaser aims at purchasing only a portion of multimedia content, 
25 there is no need for the control unit 1202 to receive the representative 
information including keywords for the corresponding semantic unit. 
However, in order for the purchaser to purchase a plurality of semantic 
unite for the multimedia content and in order for a visitor to a home page 
of the purchaser to search the purchased semantic units, the 
30 representative information needs to be input into the control unit 1202. 
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A process of inputting the representative information Into the control unit 
1202 has been described above more than one time, and thus Its 
description will be omitted. 

In step 1305, the control unit 1202 outputs execution information 

s necessary to execute the semantic unit selected by the purchaser so that 
the user can create a link between the corresponding semantic unit and 
his or her own web page in step 1305. 

Here, the execution information includes address information of 
the multimedia content purchasing system, address information of 

10 multimedia content including the semantic unit selected by the purchaser, 
and the start point and end point of the selected semantic unit 

In addition, the execution information is preferably stored in an 
encrypted execution file format so that other users cannot interpret the 
address information of the multimedia data. 

is When a visitor to the web site of the purchaser selects the linked 

execution information, the multimedia content purchasing system starts 
to operate. Then, the multimedia content purchasing system accesses 
the server where the multimedia content is stored and reproduces the 
semantic unit selected and purchased by the purchaser. 

20 In order to watch the semantic unit purchased by the purchaser, 

the visitor must use the multimedia content purchasing system. 
Therefore, the present invention makes it possible to charge the 
purchaser for the purchased semantic unit based on the number of times 
the purchased semantic unit is reproduced. 

25 The multimedia content owner can charge the purchaser for the 

purchased semantic unit in various ways through a mutual contract. For 
example, the multimedia content owner can charge the purchaser based 
on the amount of multimedia data purchased by the purchaser, the 
number of times the purchased multimedia data is reproduced, or a 

30 period of time during which the purchased multimedia data is serviced. 
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In the rate calculation unit 1204, a rate calculation method is set 
according to the mutual contract between the multimedia content owner 
and the purchaser, and rates are calculated according to the rate 
calculation method in step 1306. It is preferable to let both the 
5 multimedia content owner and the purchaser know about the calculated 
rates. 

The present invention can be realized as computer-readable 
codes written on a computer-readable recording medium. Here, the 
computer-readable recording medium includes any kind of recording 

10 medium which can be read by a computer system. For example, the 
computer-readable recording medium includes a ROM, a RAM, a 
CD-ROM, a magnetic tape, a floppy disk, an optical data storage, and a 
carrier wave (data transmission through the Internet). The 
computer-readable recording medium can be decentralized to computer 

15 systems connected over network, and a computer can read the recording 
medium in a decentralized way. 

Industrial Applicability 

As described above, the system for constructing a multimedia 

20 database and method according to the present invention and the 
multimedia database search service system and method according to the 
present invention make it possible for a user to quickly search for 
multimedia data that he or she wants and watch only necessary portions 
of the multimedia data. 

25 In addition, the system for constructing a multimedia database 

and method according to the present invention and the multimedia 
database search service system and method according to the present 
invention will enable easy trades for moving pictures through the Internet 
and contribute to activation of the moving image market. 
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